Reproducing apparatus and reproducing method

ABSTRACT

To more flexibly reproduce moving image data obtained by capturing a plurality of frames within the period of a single frame of display output while changing the exposure condition. For example, moving image data, which has been obtained by capturing a plurality of frames within the period of a single frame of display output while changing the exposure condition, is stored in a recording portion. When the moving image data is reproduced from the recording portion, a single frame is selected and output by a reproduced frame selecting portion for each period of a single frame of display output. If a user operation instructing to make the displayed image darker, in response to this user operation, a frame having an exposure condition of underexposure is selected and output from the moving image data to be reproduced from the recording portion for each period of a single frame of display output.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a reproducing apparatus and areproducing method for reproducing moving image data obtained bycapturing a plurality of images under different exposure conditions,within the period of a single frame of display output.

Description of the Related Art

In recent years, methods of capturing a plurality of images havingdifferent exposure conditions, and combining a plurality of image dataitems including the plurality of images to obtain a single image havingan expanded dynamic range have been proposed (see Japanese PatentLaid-Open Nos. 7-131718 and 2007-228099, for example). Moreover, inrecent years, with an increase in the speed at which image data is readout by an image sensing device such as a CMOS sensor, and an increase inthe resolution of such an image sensing device, it has become possiblefor an image sensing apparatus to capture images at a high resolutionand a high frame rate.

Conventionally, however, the following problem arises when ahigh-resolution, high-frame-rate moving image obtained by capturing aplurality of frames while changing the exposure condition is reproducedwithin the period of a single frame of display output. That is to say,conventionally, even though the dynamic range can be expanded bycombining a plurality of frames within the period of a single frame, itis not possible to select and reproduce a bright frame or a dark framefrom a plurality of frames within the period of a single frame asdesired by a user.

SUMMARY OF THE INVENTION

Therefore, it is a feature of the present invention to provide areproducing apparatus that is capable of more flexibly reproducingmoving image data obtained by capturing a plurality of frames within theperiod of a single frame of display output while changing the exposurecondition.

According to a first aspect of the present invention, there is provideda reproducing apparatus that reproduces moving image data obtained bycapturing images so that a plurality of frames having different exposureconditions is included in a period of a single frame of display output,the apparatus comprising: a recording unit that stores the moving imagedata; a selecting unit that selects, from the moving image data storedin the recording unit, a single frame having an exposure condition thatis designated in accordance with a user operation out of the pluralityof frames included in the period of a single frame of display output;and a reproducing unit that reproduces the frame selected by theselecting unit.

According to a second aspect of the present invention, there is provideda reproducing apparatus that reproduces moving image data obtained bycapturing images so that a plurality of frames having different exposureconditions is included in a period of a single frame of output display,the apparatus comprising: a recording unit that stores the moving imagedata; a selecting unit that selects, from the moving image data storedin the recoding unit, at least one frame out of the plurality of framesincluded in the period of a single frame of display output; areproducing unit that reproduces the frame selected by the selectingunit; and an accepting unit that accepts designation of an attentionarea with respect to the moving image data reproduced by the reproducingunit, wherein the selecting unit selects a single frame out of theplurality of frames included in the period of a single frame of displayoutput on the basis of a characteristic of the designated attentionarea.

According to a third aspect of the present invention, there is provideda reproducing apparatus that reproduces moving image data obtained bycapturing images so that a plurality of frames having different exposureconditions is included in a period of a single frame of output display,the apparatus comprising: a recording unit that stores the moving imagedata; a selecting unit that selects, from the moving image data storedin the recoding unit, at least one frame out of the plurality of framesincluded in the period of a single frame of display output; areproducing unit that reproduces the frame selected by the selectingunit; and an accepting unit that accepts designation of an attentionarea with respect to the moving image data reproduced by the reproducingunit, wherein the selecting unit selects a single frame out of theplurality of frames included in the period of a single frame of displayoutput on the basis of a characteristic of the designated attentionarea.

According to a fourth aspect of the present invention, there is provideda method for reproducing moving image data obtained by capturing imagesso that a plurality of frames having different exposure conditions isincluded in a period of a single frame of display output, the methodcomprising: a selecting step of selecting, from the moving image datastored in a recording unit, at least one frame out of the plurality offrames included in the period of a single frame of display output; areproducing step of reproducing the frame selected in the selectingstep; and an accepting step of accepting designation of an attentionarea with respect to the moving image data reproduced in the reproducingstep, wherein in the selecting step, a single frame is selected out ofthe plurality of frames included in the period of a single frame ofdisplay output on the basis of a characteristic of the designatedattention area.

According to the present invention, moving image data obtained bycapturing a plurality of frames within the period of a single frame ofdisplay output while changing the exposure condition can be reproducedin a more flexible manner.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary configuration of areproducing apparatus that is applicable to a first embodiment of thepresent invention.

FIG. 2 is a diagram showing an exemplary relationship between movingimage data saved in a recording portion and a frame to be output to adisplay portion according to the first embodiment of the presentinvention, with the horizontal axis representing time.

FIG. 3 is a flowchart showing an exemplary reproduction processaccording to the first embodiment of the present invention.

FIG. 4 is a block diagram showing an exemplary configuration of areproducing apparatus that is applicable to a second embodiment of thepresent invention.

FIG. 5 is a diagram showing an exemplary relationship between movingimage data saved in the recording portion and a frame to be output tothe display portion according to the second embodiment of the presentinvention, with the horizontal axis representing time.

FIG. 6 is a flowchart showing an exemplary reproduction processaccording to the second embodiment of the present invention.

FIG. 7 is a block diagram showing an exemplary configuration of areproducing apparatus that is applicable to a third embodiment of thepresent invention.

FIG. 8 is a diagram showing an exemplary relationship between movingimage data saved in the recording portion and a frame to be output tothe display portion according to the third embodiment of the presentinvention, with the horizontal axis representing time.

FIG. 9 is a flowchart showing an exemplary reproduction processaccording to the third embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

Hereinafter, a first embodiment of the present invention will bedescribed with reference to the drawings. FIG. 1 shows an exemplaryconfiguration of a reproducing apparatus 100 that is applicable to thefirst embodiment of the present invention. The reproducing apparatus 100includes a recording portion 101, a reproduced frame selecting portion102, a reproduction condition adding portion 103, a display portion 104,a controlling portion 105, and an operating portion 108. The controllingportion 105 is provided with a memory 106 and a CPU (central processingunit) 107. The CPU 107 controls the operation of the various portions ofthis reproducing apparatus 100 in accordance with a program stored inthe memory 106.

The recording portion 101 serving as a recording unit uses, for example,a hard disk drive, an optical disk, a nonvolatile semiconductor memory,or the like as a recording medium, and moving image data to bereproduced by this reproducing apparatus 100 is stored in the recordingportion 101. Here, it is assumed that moving image data obtained bycapturing a plurality of frames within the period of a single frame ofdisplay output while changing an exposure condition, such as exposuretime, is stored in the recording portion 101. That is to say, thismoving image data is moving image data having a higher frame rate thanthe frame rate at which video is displayed on the display portion 104,and is composed of a plurality of sets of frames having differentexposure conditions.

The display portion 104 outputs a single frame of the moving image datato a display apparatus or the like per each period of a single frame ofdisplay output. Hereinafter, the period of a single frame of displayoutput will be referred to as the period of a single reproduced frameand distinguished from the frame period of captured images. Note that inthe first embodiment, the reproducing rate in the display portion 104 isassumed to be 60 fps (frames per second).

During reproduction of the moving image data stored in the recordingportion 101, the reproduced frame selecting portion 102 serving as aselecting unit and a reproducing unit selects a frame to be reproducedout of a plurality of frames within the period of a single reproducedframe and reproduces the selected frame under the control of thecontrolling portion 105. The reproduction condition adding portion 103serving as a reproduction condition saving unit saves the reproductionstate of the moving image data stored in the recording portion 101. Forexample, the reproduction condition adding portion 103 saves informationabout the frame selected by the reproduced frame selecting portion 102.

The operating portion 108 is a portion for accepting a user operationand includes a moving image reproduction instructing portion 109 and areproduction condition setting portion 110. The operating portion 108outputs a control signal based on a user operation performed on themoving image reproduction instructing portion 109 and the reproductioncondition setting portion 110 and supplies the control signal to thecontrolling portion 105. The moving image reproduction instructingportion 109 provides an instruction to reproduce the moving image datastored in the recording portion 101. Moreover, the reproductioncondition setting portion 110 sets the reproduction condition for themoving image data stored in the recording portion 101. For example, thereproduction condition setting portion 110 can set a condition so that aframe having a predetermined exposure condition is reproduced out of aplurality of frames within the period of a single frame.

FIG. 2 shows an exemplary relationship between the moving image datasaved in the recording portion 101 and a frame to be output to thedisplay portion 104, with the horizontal axis representing time. Anexample of the moving image data stored in the recording portion 101 isindicated by reference numeral 201 in FIG. 2, and an example of themoving image data to be output to the display portion 104 is indicatedby reference numeral 202 in FIG. 2.

In this example, it is assumed that the moving image data stored in therecording portion 101 is recorded at a frame rate of 180 fps. On theother hand, the moving image data output to the display portion 104 isreproduced at a frame rate of 60 fps. That is to say, the moving imagedata stored in the recording portion 101 has been obtained by capturingthree frames per period of a single reproduced frame while changing theexposure condition. Here, it is assumed that, as indicated by referencenumeral 201 in FIG. 2 by way of example, the moving image data stored inthe recording portion 101 has been obtained by repeatedly capturingframes at exposure levels from (−1) to (+1) in the order ofunderexposure (−1), proper exposure (0), and overexposure (+1) on aframe-by-frame basis.

Note that it is assumed that information indicating the exposurecondition such as underexposure, proper exposure, and overexposure isstored in the recording portion 101 in association with each frame ofthe moving image data. For example, it is conceivable to add theinformation indicating the exposure condition to data of each frame asattribute information of the frame. This is not a limitation, and theinformation indicating the exposure condition may also be stored in therecording portion 101 as management data separate from the moving imagedata, in association with information that can specify the frame. Theinformation indicating the exposure condition is output in conjunctionwith a corresponding frame when, for example, the moving image data isreproduced from the recording portion 101. The reproduced frameselecting portion 102 can select a frame to be reproduced out of aplurality of frames within the period of a single reproduced frame onthe basis of the information indicating the exposure condition output inconjunction with each frame.

Reproducing Operation According to the First Embodiment

Next, an operation for reproducing moving image data according to thefirst embodiment will be described. For example, at time A in FIG. 2,reproduction of the moving image data stored in the recording portion101 is instructed by the moving image reproduction instructing portion109 of the operating portion 108 in accordance with a user operation.

Here, as described above, the moving image data stored in the recordingportion 101 is recorded at a frame rate of 180 fps and reproduced at aframe rate of 60 fps when output to the display portion 104.Accordingly, the reproduced frame selecting portion 102 selects oneframe having an optimum exposure condition out of every three frames ofthe moving image data stored in the recording portion 101.

Here, it is assumed that in the reproduced frame selecting portion 102,frames that have been exposed at proper exposure (0) and recorded areselected out of the frames reproduced from the recording portion 101.For example, it is assumed that unless a predetermined user operation isperformed on the reproduction condition setting portion 110, properlyexposed (0) frames are selected. In the example shown in FIG. 2,properly exposed (0) frames are selected during period T1 (see referencenumeral 203 in FIG. 2). Note that the reproduced frame selecting portion102 may selectively read out frames at a designated exposure level fromthe moving image data stored in the recording portion 101.

Next, at time B in FIG. 2, it is assumed that an instruction to decreasethe brightness of video of the moving image data that is beingreproduced is input by a user operation of the reproduction conditionsetting portion 110. In this case, the brightness of the reproducedvideo can be decreased by selecting frames at a lower exposure levelthan the frames that are currently being reproduced. In this example,the frames to be selected will be changed from properly exposed (0)frames to underexposed (−1) frames.

The controlling portion 105 instructs the reproduced frame selectingportion 102 to select underexposed (−1) frames, in accordance with thecontrol signal that is supplied from the operating portion 108 inaccordance with the user operation of the reproduction condition settingportion 110. In accordance with this instruction from the controllingportion 105, the reproduced frame selecting portion 102 selects oneunderexposed (−1) frame per period of a single reproduced frame from themoving image data to be reproduced from the recording portion 101, usingthe information indicating the exposure condition. In the example shownin FIG. 2, an underexposed (−1) frame is selected and reproduced in eachperiod of a reproduced frame during period T2 (see reference numeral 203in FIG. 2) after time B. In this case, for example, an image of a partbecomes visible where the highlights have been clipped in a properlyexposed frame.

The above-described process will be described using a flowchart. FIG. 3is a flowchart of a process for controlling the moving image datareproducing operation according to the first embodiment. First, in stepS301, content to be reproduced is selected by a user. The selection ofthe content to be reproduced is instructed by a control signal that isinput by the user operating the moving image reproduction instructingportion 109 of the operating portion 108.

After the content to be reproduced is selected, in step S302, thecontrolling portion 105 determines whether or not the content is videocontent obtained by capturing images while changing the exposurecondition. Note that although not shown in this flowchart, it is assumedthat this controlling process is performed when the frame rate of thecontent to be reproduced is higher than the display frame rate. If thecontent to be reproduced is content obtained by capturing images whilechanging the exposure condition (Yes in S302), the process is advancedto step S303. In step S303, the reproduced frame selecting portion 102selects a frame corresponding to a reproduction condition defined bydefault out of the frames of the content. For example, one frame that isgiven information indicating an exposure condition of proper exposure(0) is selected per period of a single reproduced frame. Then, theselected frames are used to perform reproduction. On the other hand, ifthe content is not content obtained by capturing images while changingthe exposure condition (No in S302), in step S311, the controllingportion 105 performs an ordinary reproduction process that is commonlyknown. In this reproduction process, one frame per period of a singlereproduced frame is selected at regular intervals and the selectedframes are displayed at the display frame rate of the display portion104. This reproduction process is not directly related to the presentinvention, and a detailed description thereof is therefore omitted.

After the content is reproduced in step S303, in step S304, thecontrolling portion 105 determines whether or not an instruction inputto increase the brightness of video that is being reproduced isgenerated by a user operation of the reproduction condition settingportion 110. If an instruction input to increase the brightness isgenerated (Yes in S304), in step S307, the controlling portion 105determines whether or not there is a frame that is given informationindicating a brighter exposure condition than a frame that is currentlybeing reproduced. If there is a brighter frame (Yes in S307), theprocess is advanced to step S308, and the controlling portion 105instructs the reproduced frame selecting portion 102 to select a framehaving a brighter exposure condition than the exposure condition of thecurrently reproduced frame. For example, in the case where the exposurecondition of the currently reproduced frame is proper exposure (0), anoverexposed (+1) frame is selected and reproduced by performing the sameprocessing as the processing of step S303.

In step S307, if the controlling portion 105 determines that there is nobrighter frame than the frame that is being reproduced (No in stepS307), the process is advanced to step S310, and the controlling portion105 displays a message stating that the brightness of the video contentcannot be changed on the display portion 104.

In step S304, if an instruction input to increase the brightness is notgenerated (No in S304), the process is advanced to step S305. In stepS305, the controlling portion 105 determines whether or not aninstruction input to decrease the brightness of the video that is beingreproduced is generated by a user operation of the reproductioncondition setting portion 110. In step S309, to which the process isadvanced if the determination result of step S305 is Yes, thecontrolling portion 105 determines whether or not there is a darkerframe than the frame that is being reproduced. If there is no darkerframe than the frame that is being reproduced (No in S309), the processis advanced to the above-described step S310. If there is a darker framethan the frame that is being reproduced (Yes in S309), the process isadvanced to the above-described step S308.

In step S306, the controlling portion 105 determines whether or not thereproduction of the content that is being reproduced is stopped. Thereproduction is stopped by the user inputting an instruction orautomatically when the end of the content is reached. If thereproduction of the content is not stopped, the processing of step S304and steps that follow step S304 is repeated to deal with the case wherethe user changes the reproduction condition of the reproduced content.If the reproduction is stopped, the above-described controlling processis stopped. The foregoing process is a controlling process for changingthe reproduction condition of the content.

Note that a frame selected by the reproduced frame selecting portion 102is supplied to the reproduction condition adding portion 103. Thereproduction condition adding portion 103 outputs the supplied frame tothe display portion 104 as it is and also performs processing for savingthe reproduction condition designated for the reproduced moving imagedata in accordance with a user operation of the reproduction conditionsetting portion 110.

As an example, a determination as to whether or not to save the state inwhich moving image data has been reproduced is made in advance by a useroperation of the reproduction condition setting portion 110. Inaccordance with the determination result, the reproduction conditionadding portion 103 adds information about a reproduced frame selected bythe reproduced frame selecting portion 102 to corresponding moving imagedata stored in the recording portion 101. It is conceivable that theinformation about the reproduced frame is saved in association with, forexample, a header or a footer of frame data in the moving image datathat is being reproduced. This is not a limitation, and the informationabout the reproduced frame can also be managed as data separate from themoving image data, in association with the frame of the moving imagedata. Moreover, the reproduction condition adding portion 103 can saveinformation that specifies a frame selected by the reproduced frameselecting portion 102 in association with the exposure condition of thatframe.

The display portion 104 outputs frames supplied from the reproductioncondition adding portion 103 as reproduced moving image data having areproducing frame rate. For example, the display portion 104 temporallyaccumulates selected frames in a buffer that is not shown, reads out theaccumulated frames in the buffer at the reproducing frame rate, andoutputs the frames as the reproduced moving image data.

In this manner, by saving the state in which moving image data has beenreproduced, when that moving image data is reproduced again, the statein which the moving image data has been reproduced by the reproducedframe selecting portion 102 can be easily replicated by referring to theinformation about reproduced frames associated with that moving imagedata on a frame-by-frame basis.

As described above, according to the first embodiment of the presentinvention, when a plurality of frames obtained by capturing imageswithin the period of a single reproduced frame while changing theexposure condition is reproduced from a recording medium, thereproduction can be performed at a brightness designated by the user.

Note that in the foregoing description, in the case where thereproduction condition is not designated by a user operation of thereproduction condition setting portion 110, a properly exposed (0) frameis selected. However, this is not limited to this example, and anunderexposed (−1) or overexposed (+1) frame may also be selected.

Moreover, in the foregoing description, the moving image data stored inthe recording portion 101 is obtained by repeatedly capturing andrecording images in three sequential states of underexposure (−1),proper exposure (0), and overexposure (+1). However, this is not limitedto this example. The moving image data recorded in the recording portion101 may also be obtained by sequentially capturing images at exposurelevels, for example, from (−3) to (+3) on a frame-by-frame basis.

Furthermore, in the foregoing description, it is assumed that therecording frame rate of the moving image data stored in the recordingportion 101 is 180 fps and the reproducing frame rate of the reproducedmoving image data output to the display portion 104 is 60 fps. However,this is not limited to this example. That is to say, when the movingimage data stored in the recording portion 101 is moving image dataobtained by capturing images at a higher frame rate than the reproducingrate and under a wider variety of exposure conditions, the range ofchoices of the exposure level at which reproduction is desired is alsoextended. Thus, the variation of the amount of change in the brightnessduring reproduction also increases, which makes it possible to changethe brightness stepwise.

Second Embodiment

Next, a second embodiment of the present invention will be described.According to the second embodiment, in the case where an attention areais selected by the user, an optimum frame is automatically selected inaccordance with the brightness of an image in that attention area.

FIG. 4 shows an exemplary configuration of a reproducing apparatus 200that is applicable to the second embodiment of the present invention.Note that in FIG. 4, portions that are common with those in thereproducing apparatus 100 shown in FIG. 1 described above are denoted bythe same reference numerals, and a detailed description thereof will beomitted.

In the reproducing apparatus 200 shown in FIG. 4, a statistic acquiringportion 301, an attention area selecting and enlarging portion 302, andan attention area selection instructing portion 303 are added to thereproducing apparatus 100 shown in FIG. 1. The statistic acquiringportion 301 acquires a predetermined statistic based on a characteristicof an image. The attention area selection instructing portion 303serving as an accepting unit provides an instruction to select anattention area in a displayed image generated from moving image datareproduced from the recording portion 101, in accordance with a useroperation of the operating portion 108. The attention area selecting andenlarging portion 302 serving as a frame generating unit selects anattention area, the selection of which has been instructed by theattention area selection instructing portion 303, and enlarges theattention area for each frame of the moving image data supplied from thereproduced frame selecting portion 102.

FIG. 5 shows an exemplary relationship between the moving image datasaved in the recording portion 101 and a frame to be output to thedisplay portion 104, with the horizontal axis representing time. Anexample of the moving image data stored in the recording portion 101 isindicated by reference numeral 501 in FIG. 5, and an example of an areain a reproduced frame selected by the attention area selecting andenlarging portion 302 is indicated by reference numeral 502 in FIG. 5.

In this example, it is assumed that the moving image data stored in therecording portion 101 is recorded at a frame rate of 180 fps and has animage size of 8000 pixels×4000 pixels. On the other hand, the movingimage data output to the display portion 104 is reproduced at a framerate of 60 fps. That is to say, the moving image data stored in therecording portion 101 has been obtained by capturing three frames perperiod of a single reproduced frame while changing the exposurecondition. It is assumed here that, as indicated by reference numeral501 in FIG. 5 by way of example, the moving image data stored in therecording portion 101 has been obtained by repeatedly capturing imagesat exposure levels from (−1) to (+1) in the order of underexposure (−1),proper exposure (0), and overexposure (+1) on a frame-by-frame basis.Note that it is assumed that information indicating the exposurecondition such as underexposure, proper exposure, and overexposure isstored in the recording portion 101 in association with each frame ofthe moving image data.

FIG. 6 is a flowchart showing an exemplary reproduction processaccording to the second embodiment. Each process step of the flowchartin FIG. 6 is performed by a program executed by the CPU 107 of thecontrolling portion 105.

Prior to the process of the flowchart in FIG. 6, reproduction of themoving image data stored in the recording portion 101 is instructed bythe moving image reproduction instructing portion 109 of the operatingportion 108 in accordance with a user operation. Here, as describedabove, the moving image data stored in the recording portion 101 isrecorded at a frame rate of 180 fps and reproduced at a frame rate of 60fps when output to the display portion 104. The reproduced frameselecting portion 102 selects one frame having an appropriate exposurecondition per period of a single reproduced frame of the moving imagedata reproduced from the recording portion 101 in response to thereproduction instruction. Here, it is assumed that properly exposed (0)frames are selected.

The frames selected by the reproduced frame selecting portion 102 aresupplied to the reproduction condition adding portion 103 via theattention area selecting and enlarging portion 302. The reproductioncondition adding portion 103 outputs the supplied frames to the displayportion 104 as they are and, in accordance with a user operation of thereproduction condition setting portion 110, performs processing forsaving a reproduction condition designated for that reproduced movingimage data in the above-described manner.

Returning to the flowchart in FIG. 6, when the reproduction of themoving image data is started, the controlling portion 105 determineswhether or not it is preset in the reproduction condition settingportion 110 that the reproduction condition is to be added and saved(step S401). If it is determined that it is set that the reproductioncondition is not to be added, the process is advanced to step S403.

On the other hand, if it is determined that it is set that thereproduction condition is to be added, the process is advanced to stepS402. In step S402, the controlling portion 105 instructs thereproduction condition adding portion 103 to add the reproductioncondition to the moving image data and save the reproduction condition.When receiving this instruction, the reproduction condition addingportion 103 adds the reproduction condition to each frame and stores thereproduction condition in the recording portion 101. The reproductioncondition can be added to, for example, a header or a footer of acorresponding frame. This is not a limitation, and the reproductioncondition can also be managed as data separate from the moving imagedata.

After the addition and saving of the reproduction condition is performedin step S402, the process is advanced to step S403. In step S403, theattention area selection instructing portion 303 selects an attentionarea in an image of a frame supplied from the reproduced frame selectingportion 102, in accordance with a user operation of the attention areaselection instructing portion 303. In the example shown in FIG. 5, anattention area is selected at time C as indicated by reference numeral505 in FIG. 5 by way of example. That is to say, when the coordinates(x, y) of an image of a frame are expressed in units of pixels, theentire image is within a range of (0, 0)-(8000, 4000). In the presentembodiment, it is assumed that an area defined by the coordinates (2000,1000)-(3919, 2079) is selected as the attention area. The coordinates ofthis selected attention area are treated as a reproduction condition ofthe moving image data. That is to say, the reproduction condition addingportion 103 serving as an area information saving unit stores thecoordinates of this selected attention area into the recording portion101 as a reproduction condition in association with the frame. Thecontrolling portion 105 supplies attention area information indicatingthe selected attention area to the statistic acquiring portion 301.

The process is advanced to step S404, and the statistic acquiringportion 301 acquires a characteristic within the attention areaindicated by the attention area information for an image of each frameof the moving image data reproduced from the recording portion 101.Here, it is assumed that a luminance characteristic, more specifically,a luminance histogram, based on luminance values of individual pixels inthe attention area is acquired. An example of the acquired luminancehistogram is indicated by reference numeral 503 in FIG. 5. In thisexample, a luminance histogram acquired for an image of a frame selectedby the reproduced frame selecting portion 102 immediately after time Cis shown. Information about the acquired luminance histogram is suppliedto the controlling portion 105.

The process is advanced to the next step S405, and the controllingportion 105 determines whether or not the exposure condition of a framethat is currently being selected is appropriate on the basis of theacquired luminance histogram. For example, the controlling portion 105determines that the exposure condition is appropriate if thedistribution of the luminance histogram of the selected attention areais not skewed either in the direction of higher luminances or in thedirection of lower luminances. Moreover, the controlling portion 105determines that the exposure condition is not appropriate if thedistribution of that luminance histogram is skewed in at least one ofthe direction of higher luminances and the direction of lowerluminances. If it is determined that the exposure condition isappropriate, the process is advanced to step S407.

On the other hand, in step S405, if it is determined that the exposurecondition of the frame that is currently being selected is notappropriate, the process is advanced to step S406. In step S406, thecontrolling portion 105 determines the lightness and darkness of theattention area on the basis of the acquired luminance histogram of theattention area and selects an exposure level corresponding to anappropriate exposure condition in accordance with the determinationresult. This can be realized by classifying the shapes of the luminancehistogram into several patterns and presetting an exposure level to beselected for each pattern. Then, the controlling portion 105 instructsthe reproduced frame selecting portion 102 to select a frame having thatappropriate exposure condition.

Specifically, in the case where video has a high pixel count for toneson the higher luminance side, there is a possibility that the highlightsare clipped in the video; therefore, a more underexposed (−1) frame thanthe frame that is currently being output to the display portion 104 isselected as a frame having an appropriate exposure condition. On theother hand, in the case where video has a high pixel count for tones onthe lower luminance side, there is a possibility that the shadows areclipped in the video; therefore a more overexposed (+1) frame than theframe that is currently being output to the display portion 104 isselected as a frame having an appropriate exposure condition.

In the example indicated by reference numeral 503 in FIG. 5, since thepixel count for tones on the higher luminance side is high, underexposed(−1) frames are selected and output to the display portion 104 aftertime D corresponding to the next frame for which the statistic isacquired (see reference numeral 504 in FIG. 5). Note that the exposurecondition selected here is treated as a reproduction condition. That isto say, the reproduction condition adding portion 103 serving as anexposure condition saving unit stores this selected exposure conditioninto the recording portion 101 as a reproduction condition inassociation with the frames.

After a frame having an appropriate exposure condition is selected, theprocess is advanced to the next step S407. In step S407, the controllingportion 105 transmits attention area information to the attention areaselecting and enlarging portion 302 and issues an instruction to enlargethe attention area in an image of the frame. In the attention areaselecting and enlarging portion 302, the attention area in the image ofthe frame supplied from the reproduced frame selecting portion 102 isenlarged on the basis of the attention area information transmitted fromthe controlling portion 105. Then, a frame composed of an image of theenlarged attention area is generated and output to the reproductioncondition adding portion 103. That is to say, the original image iscropped to the attention area, enlarged to a single-frame image size,and output as a frame.

Note that in the case where a user operation of the attention areaselection instructing portion 303 for selecting the attention area isnot performed or in the case where the selected attention area is reset,the attention area selecting and enlarging portion 302 outputs thesupplied frame as it is.

In step S408, the controlling portion 105 determines whether or not arequest to stop the reproduction of the moving image data is issued fromthe moving image reproduction instructing portion 109. For example, if arequest to stop the reproduction is transmitted from the moving imagereproduction instructing portion 109 to the controlling portion 105 inaccordance with a user operation, the controlling portion 105 stops thereproduction of the moving image data stored in the recording portion101 and terminates a series of process steps. On the other hand, if itis determined that a request to stop the reproduction is not issued, theprocess is returned to step S403. Of course, as the video proceeds, thevalues in the luminance histogram of the attention area change. For thisreason, in some cases, a frame having an exposure level that iscurrently selected may not have a proper brightness. In the presentembodiment, since processing from steps S404 to S406 is repeatedwhenever needed until the reproduction is stopped, a frame having aproper exposure condition can be selected for each frame to bedisplayed. Moreover, also in the case where the attention area ischanged during reproduction, a frame having a proper exposure conditioncan be selected and displayed by performing the above-describedprocessing.

As described above, according to the second embodiment, in the casewhere an attention area in images of a plurality of frames capturedwithin the period of a reproduced frame while changing the exposurecondition is selected, the luminance histogram of the selected attentionarea is acquired, and a frame having an appropriate exposure conditionis selected on the basis of this luminance histogram. Thus, reproductioncan be performed at a brightness suited to an image of the selectedattention area. The exposure condition of a recorded frame has beenchanged for the entire screen, and therefore, even in the case of animage of a frame that is determined to be properly exposed, a region inwhich the shadows are clipped or a region in which the highlights areclipped may be present in that frame image. Accordingly, an areaselected by the user from within the frame image is not necessarilyproperly exposed. Even in such a case, the present embodiment makes itpossible to display an image having a more appropriate brightness whenan attention area selected by the user is displayed.

Note that the higher the resolution (the larger the image size) of animage from the moving image data to be reproduced, the more it ispossible to crop and reproduce a moving image having a sufficiently highresolution even when attention is focused on an area at which that imageis present. Moreover, an editing operation can be made simple byrecording also the coordinates of the cropped area as a reproductioncondition.

Note that in the above-described first and second embodiments, a singleframe is selected out of the recorded frames and displayed; however, asingle frame image generated by combining a plurality of frames can alsobe used. For example, by using underexposed (−1) and properly exposed(0) frames, an image at an exposure level between underexposure (−1) andproper exposure (0) can be generated. In this manner, by using a processof combining recorded frames, a more precise exposure level than theactual exposure condition can be selected. In the case of the firstembodiment, the user can select the brightness more minutely. In thecase of the second embodiment, the image of the attention area can bemade an image having a more appropriate brightness.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe first and second embodiments, a frame having an intended brightnessis immediately selected and displayed in response to an instruction fromthe user or on the basis of a luminance histogram. In contrast, in thethird embodiment, the brightness of a moving image is changed stepwisetoward a target brightness by employing the above-described combiningprocess that uses a plurality of recorded frames. Thus, smooth movingimage reproduction can be realized, and the brightness of a moving imagethat is being reproduced can be changed to the target brightness. Notethat this process can be applied to the above-described first and secondembodiments. Here, a case where this process is applied to the secondembodiment will be described.

FIG. 7 shows an exemplary configuration of a reproducing apparatus 300that is applicable to the third embodiment of the present invention.Note that in FIG. 7, portions that are common with those in thereproducing apparatus 100 and the reproducing apparatus 200 respectivelyshown in FIGS. 1 and 4 described above are denoted by the same referencenumerals, and a detailed description thereof will be omitted.

In the reproducing apparatus 300 shown in FIG. 7, a frame calculationprocessing portion 601 is added to the reproducing apparatus 200 shownin FIG. 4. In the third embodiment, the reproduced frame selectingportion 102 selects a plurality of frames, that is, two or more framesper period of a single reproduced frame. The frame calculationprocessing portion 601 serving as a frame combining unit performs apredetermined calculation process, such as a cumulative addition processor a weighted average process, on the two or more frames supplied fromthe reproduced frame selecting portion 102. Then, a single frame isgenerated through this calculation process and output to the attentionarea selecting and enlarging portion 302.

FIG. 8 shows an exemplary relationship between moving image data savedin the recording portion 101 and a frame to be output to the displayportion 104, with the horizontal axis representing time. An example ofthe moving image data stored in the recording portion 101 is indicatedby reference numeral 801 in FIG. 8, and an example of the frame to beoutput is indicated by reference numeral 803 in FIG. 8.

In this example, it is assumed that the moving image data stored in therecording portion 101 is recorded at a frame rate of 1200 fps and has animage size of 8000 pixels×4000 pixels. On the other hand, it is assumedthat the moving image data output to the display portion 104 isreproduced at a frame rate of 120 fps. That is to say, the moving imagedata stored in the recording portion 101 has been obtained by capturingten frames per period of a single reproduced frame while changing theexposure condition. Here, it is assumed that, as indicated by referencenumeral 801 in FIG. 8 by way of example, the moving image data stored inthe recording portion 101 has been obtained by repeatedly capturingimages at exposure levels from (−1) to (+1) in the order ofunderexposure (−1), proper exposure (0), and overexposure (+1) on aframe-by-frame basis. Note that it is assumed that informationindicating the exposure condition such as underexposure, properexposure, and overexposure is stored in the recording portion 101 inassociation with each frame of the moving image data.

FIG. 9 is a flowchart showing an exemplary reproduction processaccording to the third embodiment of the present invention. Each processstep of the flowchart in FIG. 9 is performed by a program executed bythe CPU 107 of the controlling portion 105. Note that in the flowchartin FIG. 9, process steps that are common with those of the flowchart inFIG. 6 described above are denoted by the same reference numerals, and adetailed description thereof will be omitted. That is to say, processingof steps S401 to S405 and processing of steps S407 and S408 of theflowchart in FIG. 9 is common with that of the flowchart in FIG. 6.Moreover, since the moving image data reproduction process that isperformed prior to the process of the flowchart in FIG. 9 is the same asthe process already described in the above second embodiment, adescription thereof will be omitted here.

Hereinafter, process steps of the flowchart in FIG. 9 that are differentfrom those of the flowchart in FIG. 6 described above will be described.In step S404, a luminance histogram of a selected attention area isacquired, and in step S405, if it is determined that the exposurecondition is not appropriate, the process is advanced to step S701.

In step S701, the controlling portion 105 determines an exposure levelcorresponding to a target brightness on the basis of the luminancehistogram acquired in step S404. Next, in step S702, the controllingportion 105 determines an intermediate exposure level for display untilthe determined target exposure level is reached. For example, in thecase where the exposure level of a current frame is +1 and the targetexposure level is −1, values +0.5, 0, and −0.5 are determined asintermediate exposure levels. The determination of an intermediateexposure level can be made as appropriate on the basis of conditionssuch as the difference between the exposure level of the current frameand the target exposure level, the individual exposure condition of eachframe, the time (the number of frames to be displayed) required for thetarget exposure level to be reached, and the like.

Subsequently, in step S703, the controlling portion 105 supplies aninstruction to select frames that are used to achieve the exposure leveldetermined in steps S701 and S702 out of the frames within the period ofa single reproduced frame to the reproduced frame selecting portion 102.More specifically, the controlling portion 105 selects frames so as toachieve the intermediate exposure level determined in step S702. At thistime, in the case where the exposure condition of a single frame matchesthe intermediate exposure level, only the single frame can be selectedfrom within the period of a single reproduced frame. Moreover, in thecase where the exposure condition of a single frame does not match theintermediate exposure level, a frame corresponding to the intermediateexposure level is generated by selecting a plurality of frames fromwithin the period of a single reproduced frame and combining theseframes.

In the example shown in FIG. 8, three frames are selected out of the tenframes within the period of a single reproduced frame and cumulativelyadded to generate a single frame corresponding to the intermediateexposure level. An example of the case where the brightness of images ofgenerated frames is darkened stepwise (see reference numeral 803 in FIG.8) will be described. Note that “Level” indicated by reference numeral804 in FIG. 8 is an exposure level that is a numerical representation ofthe brightness of an image, and it is assumed that the larger thenumerical value, the brighter the image, and the smaller the numericalvalue, the darker the image.

As an example, as indicated by reference numeral 804 in FIG. 8 by way ofexample, the case where the brightness of images is decreased from anumerical value (0) to a numerical value (−3) will be considered. Inthis case, it is conceivable that the brightness of the images isdecreased stepwise, for example, from the numerical value (0) to anumerical value (−1), a numerical value (−2), and then the numericalvalue (−3). Note that it is assumed here that the numerical value (−3)corresponds to the brightness of an underexposed (−1) image, and thenumerical value (0) corresponds to the brightness of a properly exposed(0) image.

More specifically, in a period from time E to time F, a plurality offrames in such a combination that the numerical value becomes (−1) isselected. Next, in a period from time F to time G, a plurality of framesin such a combination that the numerical value becomes (−2) is selected.Then, after time G, a plurality of frames in such a combination that thenumerical value becomes (−3), which is the target brightness, isselected.

As an example, in the period from time E to time F in FIG. 8, threeframes are selected out of the ten frames within a corresponding periodof a single reproduced frame so that the numerical value becomes (−1).For example, as indicated by reference numeral 802 in FIG. 8 by way ofexample, the frames are selected in a combination of one underexposed(−1) frame and two properly exposed (0) frames. In the period from timeF to time G in FIG. 8, three frames are selected out of the ten frameswithin a corresponding period of a single reproduced frame so that thenumerical value becomes (−2). For example, as indicated by referencenumeral 802 in FIG. 8 by way of example, the frames are selected in acombination of two underexposed (−1) frames and one properly exposed (0)frame. Moreover, in the period from time G to time H in FIG. 8, threeframes are selected out of the ten frames within a corresponding periodof a single reproduced frame so that the numerical value becomes (−3).For example, as indicated by reference numeral 802 in FIG. 8 by way ofexample, three underexposed (−1) frames are selected.

Note that various methods for determining which frames are to beselected out of a plurality of frames included in the period of a singlereproduced frame are conceivable. For example, it is conceivable todetermine the frames to be selected for each of multiple consecutiveperiods of a reproduced frame so that the intervals between the selectedframes are as uniform as possible.

After the selection of frames is completed, the process is advanced tostep S704. In step S704, a plurality of frames selected within theperiod of a single reproduced frame is supplied from the reproducedframe selecting portion 102 to the frame calculation processing portion601. The frame calculation processing portion 601 cumulatively adds theplurality of frames supplied. The addition of the frames is performedby, for example, adding pixel values of pixels in a frame to be added torespective pixel values of corresponding pixels in a frame to which thatframe is added. Then, the resulting sums are divided by the number offrames added together to generate a single frame. This process forgenerating a single frame by cumulative addition of a plurality offrames is performed while resetting the result of the addition for eachperiod of a single reproduced frame. It goes without saying that in thecase where the number of frames selected within the period of a singlereproduced frame is one, the adding process is not performed.

This frame generated by the frame calculation processing portion 601using cumulative addition is supplied to the attention area selectingand enlarging portion 302. Thereafter, in step S407, processing forselecting and enlarging an attention area is performed in accordancewith a user operation, and in step S408, it is determined whether or notan instruction to stop the reproduction is issued. If it is determinedthat an instruction to stop the reproduction is issued, the operation ofreproducing the moving image data from the recording portion 101 isstopped, and a series of process steps is terminated. On the other hand,if it is determined that an instruction to stop the reproduction is notissued, the process is returned to step S403.

As described above, according to the third embodiment, when a pluralityof frames obtained by capturing images within the period of a singlereproduced frame while changing the exposure condition is reproduced, aplurality of frames is selected per period of a single reproduced frameand combined into a single frame, and then the single frame is output.Thus, the brightness can be smoothly changed to that of a target frame,and as a result, display can be performed with less discomfort.

Note that in the third embodiment, a plurality of frames is cumulativelyadded in the frame calculation processing portion 601; however, this isnot limited to this example. For example, it is also possible tocalculate a weighted average of all the frames within the period of asingle reproduced frame and generate a single reproduced frame on thebasis of the calculation results. In that case, the weight assigned toeach frame is set so that a frame having the obtained intermediateexposure level is generated from frames having each of the exposureconditions that have been input during the determination of anintermediate exposure level in step S702. In this manner, a moving imagewith more favorable image quality than that generated simply bycumulatively adding a plurality of frames can be generated.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiments, and by a method, the steps of whichare performed by a computer of a system or apparatus by, for example,reading out and executing a program recorded on a memory device toperform the functions of the above-described embodiments. For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2009-044324, filed on Feb. 26, 2009, which is hereby incorporated byreference herein in its entirety.

1. A reproducing apparatus that reproduces moving image data obtained bycapturing images so that a plurality of frames having different exposureconditions is included in a period of a single frame of display output,the apparatus comprising: a recording unit that stores the moving imagedata; a selecting unit that selects, from the moving image data storedin the recording unit, a single frame having an exposure condition thatis designated in accordance with a user operation out of the pluralityof frames included in the period of a single frame of display output;and a reproducing unit that reproduces the frame selected by theselecting unit.
 2. The reproducing apparatus according to claim 1,further comprising a reproduction condition saving unit that savesinformation that specifies the frame selected by the selecting unit intothe recording unit in association with the exposure condition that isdesignated by the user operation.
 3. A reproducing apparatus thatreproduces moving image data obtained by capturing images so that aplurality of frames having different exposure conditions is included ina period of a single frame of output display, the apparatus comprising:a recording unit that stores the moving image data; a selecting unitthat selects, from the moving image data stored in the recoding unit, atleast one frame out of the plurality of frames included in the period ofa single frame of display output; a reproducing unit that reproduces theframe selected by the selecting unit; and an accepting unit that acceptsdesignation of an attention area with respect to the moving image datareproduced by the reproducing unit, wherein the selecting unit selects asingle frame out of the plurality of frames included in the period of asingle frame of display output on the basis of a characteristic of thedesignated attention area.
 4. The reproducing apparatus according toclaim 3, further comprising a frame generating unit that generates aframe composed of an image of the attention area from an image of theframe selected by the selecting unit on the basis of the characteristicof the attention area, wherein the reproducing unit reproduces the framegenerated by the frame generating unit.
 5. The reproducing apparatusaccording to claim 4, wherein: the selecting unit selects two or moreframes out of the plurality of frames in the period of a single frame ofdisplay output, the apparatus further comprises a frame combining unitthat combines the two or more frames selected by the selecting unit togenerate a single combined frame, and the frame generating unitgenerates a frame using an image in an area of the combined framegenerated by the frame combining unit.
 6. The reproducing apparatusaccording to claim 5, wherein the selecting unit selects the two or moreframes out of the plurality of frames in such a combination that thecharacteristic changes stepwise for each period of a single frame ofdisplay output.
 7. The reproducing apparatus according to claim 3,wherein: the characteristic is a luminance characteristic in theattention area, and the selecting unit selects an exposure condition onthe basis of the luminance characteristic and selects a frame on thebasis of the selected exposure condition.
 8. The reproducing apparatusaccording to claim 7, further comprising an exposure condition savingunit that saves information that specifies the frame selected by theselecting unit into the recording unit in association with the exposurecondition of the frame.
 9. The reproducing apparatus according to claim3, further comprising an area information saving unit that savesinformation indicating the attention area into the recording unit inassociation with the frame of the image in which the attention area isdesignated.
 10. A reproducing apparatus that reproduces moving imagedata obtained by capturing images so that a plurality of frames havingdifferent exposure conditions is included in a period of a single frameof output display, the apparatus comprising: a recording unit thatstores the moving image data; a selecting unit that selects, from themoving image data stored in the recoding unit, at least one frame out ofthe plurality of frames included in the period of a single frame ofdisplay output; a reproducing unit that reproduces the frame selected bythe selecting unit; and an accepting unit that accepts designation of anattention area with respect to the moving image data reproduced by thereproducing unit, wherein the selecting unit selects a single frame outof the plurality of frames included in the period of a single frame ofdisplay output on the basis of a characteristic of the designatedattention area.
 11. (canceled)